Digital microscope slide scanning system and methods

ABSTRACT

Provided herein are systems methods including a design of a microscope slide scanner for digital pathology applications which provides high quality images and automated batch-mode operation at low cost. The instrument architecture is advantageously based on a convergence of high performance, yet low cost, computing technologies, interfaces and software standards to enable high quality digital microscopy at very low cost. Also provided is a method based in part on a stitching method that allows for dividing an image into a number of overlapping tiles and reconstituting the image with a magnification without substantial loss of accuracy. A scanner is employed in capturing snapshot images. The method allows for overlapping images captured in consecutive snapshots.

CROSS-REFERENCE

This application is a divisional application of U.S. patent applicationSer. No. 12/054,309, filed Mar. 24, 2008, now U.S. Pat. No. 8,098,956,which claims the benefit of U.S. Provisional Application No. 60/896,832,filed Mar. 23, 2007, and U.S. Provisional Application No. 60/896,852,filed Mar. 23, 2007, the contents of which are incorporated herein byreference in their entirety.

BACKGROUND OF THE INVENTION

Molecular imaging-Identification of changes in the cellular structuresindicative of disease remains a key to the better understanding inmedicinal science. Microscopy applications are applicable tomicrobiology (e.g., gram staining, etc.), Plant tissue culture, animalcell culture (e.g. phase contrast microscopy, etc.), molecular biology,immunology (e.g., ELISA, etc.), cell biology (e.g., immunofluorescence,chromosome analysis, etc.) Confocal microscopy: Time-Lapse and Live CellImaging, Series and Three-Dimensional Imaging.

There have been advances in confocal microscopy that have unraveled manyof the secrets occurring within the cell and the transcriptional andtranslational level changes can be detected using fluorescence markers.The advantage of the confocal approach results from the capability toimage individual optical sections at high resolution in sequence throughthe specimen. However, there remains a need for systems and methods fordigital processing of images of pathological tissue that provideaccurate analysis of pathological tissues, at a relatively low cost.

SUMMARY OF THE INVENTION

The present invention provides method for reconstituting an imagecomprising dividing an image into a number of overlapping tiles capturedin consecutive snapshots and reconstituting the image with amagnification without substantial loss of accuracy. A digital scanner isemployed in capturing snapshot images.

In one embodiment the method comprises calculating stitch points betweentwo consecutive snapshots acquired by the scanner for stitching adjacenttiles. Images have a common overlap of at least N pixels between themwherein N is greater than 1 pixel.

In one embodiment, the overlap between two adjacent tiles is up to 30%of the total pixels in each tile.

In another embodiment, the overlap between two adjacent snapshots is upto 25%, up to 20%, up to 15%, up to 10%, up to 5%, or up to 1% of thetotal pixels in each tile.

In yet another embodiment the overlap between two adjacent snapshots isup 1% of the total pixels in each tile.

In yet another embodiment the method of the invention does not requirealignment between adjacent snapshots.

In a further embodiment the method allows more flexibility in motioncontrol while taking the snapshots.

In a still further embodiment stitching involves a step with an inputcomprising a set of two images and an output comprising stitch pointsfor stitching the two images.

In yet another embodiment the stitching is carried out through cornerpoints selected in overlapping regions of the two tiles to be stitched.The step comprises determining correspondence between corner points in aminimum overlap region, for example in the left/top (herein afterreferred as Tile1) and maximum overlap region in the right/bottom(herein after referred as Tile2). The search region around the cornerand the number of corners are iteratively increased until a desiredmatch is achieved. The strength of the correspondence computation may beenhanced by employing schemes such as voting, and histograms ofdisplacement values. A confidence value is associated with the computedstitch displacement. The confidence may be computed using methods thatinclude but are not limited to statistical measures and crosscorrelation values,

In yet another embodiment the stitching may be carried out based onfinding the difference between an overlap region in the two tiles. Forexample, a method for computing the difference includes, withoutlimitation, measuring difference of pixel values along two correspondinglines of the two image tiles. This is based in part on the observationthat the pixel difference would be minimum along the line when the linesmatch in the two tiles. The method may include selecting a candidateline in the first tile and searching for the corresponding best match inthe second tile. The search may be restricted to the region of expectedoverlap in the second tile. The search space includes both horizontaland vertical directions. The optimal point is found where the mean pixeldifference is minimized. The accuracy of the method in enhanced methodby considering a multitude of candidate lines in the first tile andfinding the best match in the second tile. A consensus is measured usingall candidates. A measure evaluates the goodness of the candidate linesin the first tile. The goodness measure includes but is not restrictedto using the variance of the pixels in the line.

In yet another embodiment the stitching may be carried out based oncross-correlation using multiple sized windows between the overlappedregions of the two tiles to be stitched. Candidate windows are selectedin the first image based on goodness values. A measure of goodness valueincludes but is not limited to the contrast or variance of the pixelswithin the window. Candidate matching windows in the second tile areiteratively found by first searching for the candidate using smallerwindows and then increasing the size of the windows for more accurateestimates. For each window location on the first tile, multiplecandidates are found in the second tile which are qualified and rejectedin multiple iterations. The search is extended by iteratively addingcandidate windows in the first tile.

In a further embodiment the stitch points identify the area of overlapbetween adjacent images wherein the overlap area is cropped from one ofthe images such that when the adjacent images are put together theentire scene is rendered without loss of information.

In a further embodiment the images are not transformed by blending,warping and there is no significant loss of image information.

In a further embodiment the pixels in the overlap region betweenadjacent tiles are stored as the “seams” of the stitching.

In still a further embodiment the output produces an image at leastequivalent to a tiling system that takes an image and through motioncontrol moves the high resolution camera such that the image boundary isaligned with no or negligible overlap of images.

Another embodiment of the invention provides a method for correcting ascanned image.

In one embodiment the correction employs a flat field correctionalgorithm.

In yet another embodiment the method employs air-blank or glass-blankimage in conjunction with solving non-uniform illumination andabrasions.

In a further embodiment the correction step comprises:

-   -   capturing a glass blank image or air-blank image;    -   smoothing the image using Gaussian filter (e.g.; radius 10);    -   dividing each pixel in image by Max of R, G and B plane values        and then reciprocating the image;    -   calculating a multiplying factor for R, G and B planes for each        pixel; and    -   multiplying all the pixels in the given image with multiplying        factor determined from the blank image.

One embodiment provides a method for reconstituting an image comprisingdividing the image into a number of overlapping tiles captured in aseries of snapshots and reconstituting the image with a magnificationwithout substantial loss of accuracy; wherein the method comprises:

-   -   capturing snapshot images through a digital scanner;    -   calculating stitch points between two consecutive snapshots        defining a first tile and a second tile, respectively; wherein        the first and second tile images have a common overlap of at        least N pixels between them; wherein the method further        comprises:        -   a. detecting and recording corner points in a minimum            overlap region and a maximum overlap region, whereby            detected corner points or a selected subset of corner points            are sorted and maintained in a list;        -   b. for each corner point selected in the first tile,            determining a set of possible stitch points in the second            tile; wherein candidate points are selected using a goodness            criterion;        -   c. maintaining a pair of stitch points that have a matching            score that is greater than a defined or computed threshold;        -   d. computing a displacement between the tiles;        -   e. calculating a confidence score associated with the            computed displacement; and        -   f. determining a stitching point for stitching the first and            second tiles displaying or showing the stitch image for            further analysis.

Another embodiment provides a method wherein stitch points identify anarea of overlap between tile 1 and tile 2.

Another embodiment provides a method further comprising cropping theoverlap area from one of the image tiles such that when the first andsecond tiles are put together an entire scene is rendered withoutsignificant loss of information.

Yet another embodiment provides a method further comprising correcting ascanned image by:

-   -   a. capturing a glass blank image or Air-blank image;    -   b. smoothing the image;    -   c. dividing each pixel in the image by Max of R, G and B plane        values and then reciprocating the image;    -   d. computing a multiplying factor for R, G and B planes for each        pixel; and    -   e. multiplying all the pixels in the given image with        multiplying factor determined from the blank image to apply        flat-field correction to the given image and displaying or        storing the corrected image for further analysis.

Another aspect of the invention provides a slide rack auto-loader foruse in a digital microscope slide scanning system comprising a userremovable rack comprising multiple slide holders.

In one embodiment the holders hold 20 slides per basket and are arrangedwith two stacked baskets in four columns.

In another embodiment the slide autoloader further comprises areflective, IR sensor, for example, at a distance of few millimetersfrom the end of the slides.

In yet another embodiment the slide rack auto-loader further comprises adetector which looks at light scattered from the slide.

In yet a further embodiment the slide rack auto-loader comprises fourlaser/detector pairs to scan down each of the four rows of slides.

In still another embodiment, as the slide rack is translated vertically,each slide is sensed and the position recorded.

In a further embodiment the positions are used for:

-   -   determining which slides are present for scanning and/or    -   determining the required vertical height to safely extract the        slide from the rack for scanning. In still another embodiment        the invention provides a slide rack autoloader comprising:

low cost, non-contact slide detector sensor system to determine whichslides are present;

sensor system to accurately determine slide position for safe slideextraction from rack to scanning system; and

set of standard slide racks for compatibility to other slide processingequipment.

BRIEF DESCRIPTION OF THE DRAWINGS

Features of the invention are set forth with particularity in theappended claims. A better understanding of the features and advantagesof the present invention will be obtained by reference to the followingdetailed description that sets forth illustrative embodiments, in whichthe principles of the invention are utilized, and the accompanyingdrawings of which:

FIG. 1 shows a tile translated in the XY plane according to oneembodiment of the invention;

FIGS. 2 and 3 show a slide rack according to one embodiment of theinvention;

FIG. 4 shows how a detector looks at light scattered from a slide;

FIG. 5 a shows a slide from an end on view in a properly loadedcondition. FIG. 5 b shows the resulting detected laser pulse wherein thewidth of the pulse z_(s) is compared to reference values t₁ and t₂;

FIG. 6 a shows two slides in adjacent slots presented in an end on view.FIG. 6 b shows the resulting detected laser pulses;

FIGS. 7 a, 7 b, 8 a and 8 b illustrate how error conditions aredetected. A double-slide condition is shown in FIG. 7 a along with thedetector signals shown in 7 b and a tilted slide condition is shown inFIG. 8 a along with the detector signals shown in 8 b.

DETAILED DESCRIPTION OF THE INVENTION

In one embodiment, the invention provides a design of a microscope slidescanner for digital pathology applications which provides high qualityimages and automated batch-mode operation at low cost. High qualityoptics and high precision mechanical systems are generally veryexpensive, and existing products that address this need are pricedaccordingly. The instrument architecture implemented according to theinvention is advantageously based on a convergence of high performance,yet low cost, computing technologies, interfaces and software standardsto enable high quality digital microscopy at very low cost.

The invention is based in part on a stitching method that allows fordividing an image into a number of overlapping tiles and reconstitutingthe image with a magnification without substantial loss of accuracy. Ascanner is employed in capturing snapshot images. The method allows foroverlapping images captured in consecutive snapshots. One embodimentcomputes the stitch points between two consecutive snapshots acquired bythe scanner. These images have a common overlap of at least N pixelsbetween them. N is greater than 1 pixel. The method of the inventionallows for significant overlap between consecutive snapshots. The methodprovides reconstituted images that are a substantially accuratemagnification of an original slide with overlaps of up to 30% of thetile. That is, if each snapshot comprises 1000 pixels, the overlapbetween two adjacent snapshots can be up to 300 pixels. The methodprovides results of increasing accuracy with overlaps between twoadjacent snapshots of up to 25%, up to 20%, up to 15%, up to 10%, up to5%, or up to 1%. A 1% overlap between two adjacent snapshots containing1000 pixels each is 10 pixels, which is significantly more than overlapallowed by conventional methods. That is, the invention providessubstantially accurate magnified renderings while allowing significantoverlap between adjacent snapshots.

Allowing a substantial overlap (significantly more than 1 pixel) betweenadjacent snapshots in the order of 1% or more of the total pixels in thetile provides great flexibility in the design of the scanner.Embodiments of the invention do not require the levels of accuracy inpositioning the snapshots required by conventional methods. There is noneed to require alignment between adjacent snapshots. In addition,allowing for overlap and providing an accurate method for processingoverlapping snapshots provides more flexibility in motion control whiletaking the snapshots.

In one embodiment, the input to the stitching method is a set of twoimages and output is the stitch points to be used for stitching the twoimages to form an image combining the features contained in the twoimages or image tiles. The steps involved include the following—

-   -   a. Detect and record corner points in the minimum overlap region        in the left/top (Tile1) and maximum overlap region in the        right/bottom (Tile2). All detected corner points or a selected        subset of corner points are sorted and maintained in a list.    -   b. For each corner point selected in Tile 1, the method        determines the set of possible stitch points in Tile 2. The        candidate points are selected using a goodness criterion that is        based on but not restricted to the tolerance band around each        point in tile 1.    -   c. Maintain the pair of stitch points that have a matching score        that is greater than a defined or computed threshold. An example        matching score is the cross correlation of the intensity values        around the corner point. Other matching scores may be used and        are contemplated to be with the scope of the invention.    -   d. Robust methods are used for computing the final displacement        between the tiles. These methods include but are not limited to        using the votes associated with each candidate displacement. For        example, the displacement histogram of all the corner points of        tile 1 computes the votes. A displacement getting the most votes        and satisfying an acceptance criterion is chosen as the final        displacement.    -   e. A confidence score is associated with the computed        displacement. An example confidence measure is the cross        correlation values. Alternative statistical or non statistical        methods can also be used.    -   f. If a robust displacement estimate is not available then the        computation is repeated by iteratively increasing the search        region and/or relaxing the goodness criterion of step b above.    -   g. If the algorithm still fails to get the stitch point return        the error code indicating stitching failure.

In one embodiment, the stitch points identify the area of overlapbetween adjacent images. This overlap area is cropped from one of theimages such that when the adjacent images are put together the entirescene is rendered without significant loss of information. The imagesused in this operation are not transformed by blending, warping, etc.,therefore there is no significant loss of image information. The outputof this operation produces an image at least equivalent to a tilingsystem that takes an image and through complicated motion control movesthe high resolution camera such that the image boundary is aligned(within one pixel) with negligible overlap of images. For example, for aconventional system utilizing a 20× microscope objective, the pixel sizeis about one-half of a micron. A rule of thumb for specifying theresolution of the motion control would be 5-10× better than the accuracya conventional method would be attempting to achieve, therefore themotion control system required by such conventional method would beabout 0.05 to 0.1 micron. Use of a 40× scanner would require resolutionsof the motion control system to 0.025 to 0.05 micron. For example, theAperio Scanscope T2 utilizes a Renishaw RGH24G tape scale opticalencoder with a resolution of 0.05 or 0.1 micron. By contrast, the methodof the invention does not require such high level of accuracy in motioncontrol system, allowing the cost of production of the scanner to begreatly reduced.

The methods of the invention allow for the pixel values in the overlapregions between the tiles that are not used in the final composed imageto be saved. The saved seams are used for evaluating the quality of thestitching and for correcting any introduced errors.

The methods of the invention allow for obtaining images of high accuracywhile providing much flexibility in the scanner design. Coupling themethods of the invention with high performance PCs, with fastprocessors, large amounts of memory, and very high capacity disk drives,which today can be obtained at very reasonable cost enables the systemdesigner to relax the specifications and performance requirements forthe optical and mechanical subsystems without compromising the qualityof the images obtained. Since these subsystems are typically the mostexpensive parts of a digital microscope, the total cost of the systemcan be greatly reduced with the approach provided by the presentinvention.

In another embodiment the invention provides a method for correcting ascanned image. In this embodiment, a flat field correction algorithm isemployed for correcting the image using air-blank or glass-blank imageand solving non-uniform illumination and abrasions.

In one embodiment, the method comprises the following steps:

-   -   a. Capturing a glass blank image or Air-blank image.    -   b. Smoothing the image, for example, by using Gaussian filter        (radius 10)    -   c. Dividing each pixel in the image by Max of R, G and B plane        values and then reciprocating the image.    -   d. Computing multiplying factor for R, G and B planes for each        pixel.    -   e. Multiplying all the pixels in the given image with        multiplying factor determined from blank image to apply        flat-field correction to the given image.

The image correction method according to the invention is advantageousin that it allows for solving the non-uniform illumination observed onthe acquired images. For example, the image acquired may be rectangular(due to the CCD design) and the illumination being projected on theslide for acquiring image may be circular as it comes through theobjective.

The present invention also provides method for finding a region ofinterest in a slide of a tissue using a low resolution (thumbnail) imageof the whole tissue in the slide to help a scanner to start scanning theslide only in the region of interest and thus avoids the unnecessaryscan of the entire slide. In one embodiment, the thumbnail image iscapture in a single shot.

An embodiment of the invention segments the thumbnail image intodifferent regions of the slide and identifies the different regionsincluding the region that corresponds to the tissue area.

A further embodiment first segments and identifies the label region ofthe glass slide image. Most pathology glass slides have a label attachedon the slide. These regions have the barcode and other identifyinginformation printed on a label. In some cases handwritten markings areleft on the glass. All such label regions are segmented and identifiedby finding the bounding box that encloses all the information.

An embodiment of the invention identifies the cover-slip boundaries onthe glass slide. The tissues are usually placed under a glass coverslipon the slide. The four boundaries of the coverslip are straight linesthat are usually aligned to the horizontal or the vertical. However,they may also be at an angle. The invention detects the presence of suchstraight lines and combines the information to detect the cover slipboundary.

An embodiment of the current invention also detects the stainingartifacts that appear at the edges of the coverslip and the slides.

An embodiment of the invention detects the tissue area within the regionleft out after the label, staining artifacts, and the cover slipboundaries are detected. Within the remaining area the method segmentsthe pixels into tissue vs. non-tissue. The steps involve the automaticdetection of an intensity threshold that separates tissue pixels fromthe non tissue pixels. Following the detection of a kernel region oftissue, the region is grown to incorporate the entire tissue area. Thegrowing process uses automatically determined hysterisis thresholds thatallow light tissue also to be included into the regions. Smallhypothesized areas are removed as they usually correspond to dirt orartifacts on the slide. As a final step all the detected tissue areasare merged into a larger region of interest for the scanning process.

In one embodiment, the invention provides a method for finding a regionof interest using the thumbnail image of the whole tissue in the slide.This helps the scanner to start scanning the slide only in the region ofinterest and thus avoids the unnecessary scan of the entire slide.

The thumbnail image consists of a representation of the glass slide as awhole. This includes the label and the actual tissue spread on the glassslide.

Acquiring the thumbnail image has the following benefits.

The label from the thumbnail image can be used an identifier. The labelimage may have a barcode specifying the details of the case, stain typeused etc. It may also have label information that may not include abarcode. This can be used as a cross verification tool to correlate theglass slide with the actual image.

The thumbnail image also gives a very good indication of the tissuespread on the glass slide. The tissue spread on the glass slide isdetermined by the algorithm of the invention.

The method of the invention allows for saving time while scanning theglass slide, as only the area with the tissue needs to be imaged and thesurrounding empty glass slide need not be scanned.

The method of the invention also allows better identification of focuspoints. This helps selection of a proper auto focus algorithm based onthe layout of the tissue on the slide.

The embodiment of the invention detects the different regions within theslide and identifies the region corresponding to the tissue area ofinterest. The steps involved in the detection of region of interest are

-   -   a. Identify regions containing characters written on the slide.        Sometimes there are characters written near labels on the slide.        This step first determines the presence of such characters near        the label and then tries to identify the region that contains        these characters. The detection procedure identifies such        regions by a combination of geometrical and color information.        Remove the identified regions for searching the tissue region of        interest.    -   b. The dark black regions (like black label strip and corner        regions) are also removed from the ROI input gray image. A        combination of geometric and color information is used to        robustly detect such regions.    -   c. Identify cover slip lines in the Thumbnail AOI (Area of        Interest) input image. A line detector that is biased towards        horizontal lines is used for the detection. An example of such a        method, but is not restricted to, uses the Hough transform. A        set of horizontal edges magnitude peaks are identified in the        regions of the images where the cover slip is expected to be        present e.g. Lower part (0 to Image_Height/6) and Upper part        (Image_Height to Image_Height/3) of the image. The number of        such peaks is also a program parameter or is selected based on        empirical evidence. A validation step verifies that the detected        lines define a valid cover slip region. The detected area is        then used to update the bounding box within which the tissue is        searched.    -   d. One embodiment for the identification of tissue region within        search bounding box consists of        -   i. Processing the input RGB image in the L*ab color space.        -   ii. Enhancing the contrast in the L (Luminosity) channel.        -   iii. Smoothing the image by applying an appropriate            smoothing filter. An example of such a filter is a gaussian            with a parameterized kernel size.        -   iv. Automatically computing a threshold for the pixel            intensity such that the tissue pixels are distinguished from            the background pixels. One such method for automatic            detection comprises finding the large change in the            histogram. Compute the histogram and find Maximum of the            derivative. Consider only the middle span of the histogram.            The threshold value is a function of the detected change in            histogram point.        -   v. Another method of automatically computing the threshold            is by using a Mixture of Gaussian model for the foreground            pixels.        -   vi. Yet another method of automatically computing the            threshold is the use of OTSU method/algorithm.        -   vii. Yet another method of automatically computing the            threshold is the use of k-means clustering on the intensity            values.        -   viii. Threshold image using the values computed and segment            into foreground and background.        -   ix. Fill Holes in the segmented objects and then grow the            foreground region. The region growing procedure uses a            second threshold parameter computed from the image data and            a hyperparameter set based on empirical data.        -   x. Another method of automated threshold selection for            region growing distinguishes between whole tissue slides            (WS) and tissue microarray (TMA) slides. The distinction            between TMA and WS is based on using the feature that the            TMA has a regular grid of tissue samples placed on the            slide. The regularity of grid feature is detected using the            frequency spectrum information of the image.        -   xi. Final optional post processing is done using            Morphological opening to smooth borders.    -   e. Identify the regions of interest using the segmented image:        -   i. Label the segmented objects to get the area and bounding            box information for each object.        -   ii. Combine overlapping object AOIs if any.        -   iii. Reject smaller objects or objects spanning across the            image from left to right with a very small height as these            are cover slip artifacts in the image. Reject edge connected            objects with high aspect ratio.    -   f. Delete small AOIs identified in the image as these are small        artifacts in the image.

The invention also provides scanning imaging systems with features thatenhance image analysis when combined with the image processing methodsdisclosed herein.

In one embodiment, the invention provides a scanning imaging system witha flexible and adaptive illumination.

Illumination: Brightfield microscopy typically uses Kohler illumination,which is a commonly used technique that provides even illumination overthe desired field of view. In Kohler illumination, each point in thefield of view is illuminated by a “cone” of light, with a cone angledesigned to match the numerical aperture of the microscope objective.The illuminated area is also matched to the field of view of theobjective. Kohler illumination typically requires a powerful lightsource, because to achieve uniform illumination, the light source isimaged into the sample plane at very high magnification, which meansthat most of the light source energy is rejected by the illuminationsystem. Such light sources are expensive, require large power supplies,generate significant heat, and also have very short lifetimes.

To enable low-cost digital microscopy for pathology, it is desirable touse solid-state light sources such as

LEDs. These require very low power, which reduces the need for largepower supplies and heat sinks, and also have long lifetimes, reducingsystem maintenance costs. Another advantage of LED illumination isstable spectral output. With lamps, the color spectrum changesdrastically with intensity, but this is not the case with LEDs.Microscopes using incandescent lamps often use blue filters in theillumination path, to remove the yellowish tint. With LEDs, this is notrequired.

The invention provides an illumination system that can mimic Kohlerillumination system but uses white LEDs, with the followingcharacteristics:

-   -   a. Commercially available, inexpensive white LED, consisting of        a blue LED die, covered with phosphor coating to broaden the        spectrum through the visible region.    -   b. Adjustable illumination area, to match the field of view of        the objective.    -   c. Adjustable numerical aperture (up to max of 0.5 NA), to        appropriately fill and match the numerical aperture of the        microscope objective.    -   d. Very low power, typically 50 to 100 mW (versus 10 to 100        Watts with typical lamps)    -   e. Approximately 20% variation in illumination intensity across        the field of view, corrected (flat-fielded) by software        calibration techniques.

Imaging system: The design described herein can be implemented using aconventional 20×/0.5 NA infinity-corrected microscope objective, that isavailable from several manufacturers. Together with a tube lens, itprojects an image of the sample on to a CCD sensor, with pixel count of1024 by 768, and 4.65 micron pixel pitch. To achieve image pixels withapproximately 0.5 micron spacing (which is approximately the same as theresolution of the objective), we need 10× magnification from the sampleto the CCD. This is achieved by using a tube lens with focal length of90 mm, which is ½ the focal length of the 180 mm tube lens used in astandard microscope. With this magnification, each image captured by thecamera covers a rectangular field of 0.512 mm by 0.384 mm. The inventionis not limited to the 1024×768 CCD sensor. For example, with appropriatechoice of tube lens, higher density arrays such as 1032×776 or 1392×1032or 1624×1224 can be used to increase the rectangular field at the objectplane. Choosing higher density, larger CCD arrays, further decreases thescan time by acquiring fewer frames while maintaining opticalresolution. The trade-off with increasing array size is the potentialdifficulty in correcting focus in the frame area if the biologicaltissue varies widely in height.

Coupled with the illumination system described above, the requiredexposure time to saturate a high-quantum-efficiency CCD is approximately500 microseconds. Also, since it is possible to drive the LED withsignificantly higher power, up to 1 or a few Watts, it is possible toreduce the exposure time down to approximately 10 to 50 microseconds

In normal operation, the LED current and/or CCD exposure time isadjusted before starting a scan, so that an image of a “blank” samplewill result in nearly a full-scale, or “saturated” signal on the CCDcamera. While scanning, the LED current and CCD exposure time remainfixed at those values.

Slide-scanning operation: To scan an entire microscope slide with a“tiled image” architecture (see FIG. 1), the slide is translated in Xand Y (the focus axis is in Z), pausing at each frame location while animage is acquired. Using the camera and imaging system described above,to scan an area of 15 mm by 15 mm requires approximately 1400 imageframes. (This number allows for some overlap between adjacent frames,which is required to adaptively stitch the frames together.) To achievereasonable scan speeds of approximately 5 minutes per microscope slide,it is required to move very quickly from one frame to the next, focusthe objective in that location, acquire the focused image, then move tothe next frame and repeat the process. The stitching methods of theinvention allow for reduced motion control, for example higher motionspeed, yet preserve the accuracy of the processed slides.

An additional benefit of using LEDs relates to the short duration, highbrightness exposure which allows an alternative method of scanningcomparable to “on-the-fly” scanning. Whereby the stage is commanded toexecute a constant velocity move and the camera acquires images at atiming that allows coverage, minimal overlap of one frame to the next,of the complete area of interest on the slide. At 500 microsecond andlower, no image blur is encountered at the appropriate choice of stagevelocity. By eliminating the step and settle times associated with theaforementioned step-and-repeat method scan times can be further reduced.

Another aspect of the invention relates to the flexibility provided inthe vibrations allowed while taking the snapshots. For tiled imagingsystems, it is critical that system vibrations be minimized and allowedto dampen out before the frame image is captured. Vibration amplitudesthat are more than about 1/10 of a pixel would cause blurring of theimage. However, any delays to allow vibrations to dampen can increasethe scan time considerably. For example, a 200 ins delay at each frame,multiplied by 1400 frames, would add over 4 minutes to the total scantime for a 15 mm by 15 mm region.

By providing a very bright illumination source coupled with a verysensitive detector, it is possible (as described above) to captureimages with very short exposure times. If the exposure time is shortenough relative to the vibration velocity of the sample, it isacceptable to capture the image before the vibrations have dampened tosub-pixel amplitudes. Effectively, the short exposure is acting as astrobe light to freeze an image of the moving sample. The disadvantageto this technique is that the image is captured at an unknown phase ofthe vibration ring-down process. The result is that the XY location ofthe sample is not precisely known, and a software technique will berequired to adaptively stitch adjacent images together at the correctrelative positions. The stitching techniques described herein allow theuser to overcome this problem.

This technique can be extended to use feedback from electronicacceleration sensors, to accurately time the image acquisition to occurat a certain phase in the vibration ring-down profile. For example, theimage could be acquired at a peak of vibration excursion, when thevibration velocity is near zero. Another mode of operation would be tocapture the image at a zero-crossing of the measured acceleration, whichwould allow for image acquisition with the minimized vibrationalposition uncertainty. The accelerometer can also be used to detect whenexternal vibrations, such as a user bumping against the instrument, mayhave caused imaging blurring. In this case, the portion of the imagethat is suspect can be automatically re-scanned, or a warning can bedisplayed to the user.

The embodiments described here allow for a low-cost, high speed, tiledimaging system. Other features of the system of the invention include:

-   -   a. bright light source    -   b. imaging detector with high quantum efficiency    -   c. image capture without regard to vibrations, both        internally-generated and externally-generated.    -   d. software algorithms to correct for position uncertainty due        to vibration    -   e. optionally, the invention may use feedback from acceleration        sensors to fine-tune the timing of image capture to minimize        blurring, or to minimize position uncertainty due to vibration.

In another embodiment, the invention provides a slide auto-loadersystem: The design of a microscope slide scanner for digital pathologyapplications generally requires the user to scan many specimen slides ina high throughput fashion. Consequently, the instrument must allow theuser to load many slides at a single time. A slide auto-loader systemshould:

-   -   a. Detect all available slides in the system without user input.    -   b. If possible, use off-the-shelf slide racks for compatibility        with prior and subsequent processing equipment for ease of use.    -   c. Detect and select slides improperly positioned in the slide        rack to ensure no slide breakage.

FIGS. 2 and 3 illustrate slide rack auto-loader systems as describedherein. Other system configurations are also contemplated.

In one embodiment according to the invention, the slide rack auto-loadercomprises a user removable rack that contains multiple commercial slideholders. In this embodiment, the commercial racks are Sakura baskets(Part #4768) holding 20 slides per basket and arranged with two stackedbaskets in four columns. The assembly is shown below. A reflective, IRsensor such as Fairchild # QRE00034 is placed a few millimeters from theend of the slides. Alternatively, an inexpensive laser diode, such asthose used on laser pointers can be used to obtain greater selectivityof the slide edge. This laser is monitored by a detector which looks atlight scattered from the slide. This is shown schematically in FIG. 4.In one embodiment, the invention uses 4 laser/detector pairs to scandown each of the 4 rows of slides. As the slide rack is translatedvertically, each slide edge reflects laser light to the detector. Thedetector produces an analog voltage change which is fed to an A/D on amicroprocessor. The shape of the reflected light pulse can be processedin terms of width of pulse and height of pulse to determine if a slideis in the slide basket slot or not. Furthermore, the shape of thereflected light pulse can be used to determine if two slides are loadedinto the same slot or a slide is tilted within the slot. Thisinformation is useful to prevent mishandling or damage of slide by theslide grabber mechanism FIG. 5 a shows a slide from an end on view in aproperly loaded condition. FIG. 5 b shows the resulting detected laserpulse wherein the width of the pulse z_(s) is compared to referencevalues t₁ and t₂. If z_(s) falls between the reference values slidepicking continues, if z_(s) falls either below or above the referencevalues an error condition is produced.

Referring to FIG. 6 a two slides are presented in adjacent slots againpresented in an end on view. The pulse widths are compared as explainedabove and the spacing or pitch of the pulse widths are compared toadditional reference value z₁. If the pitch of pulse widths are notwithin z₁ of a multiple of fixed slide positions an error is produced.Two important error conditions are detected with this methodology, adouble-slide condition shown in FIG. 7 a along with the detector signalsshown in 7 b and a tilted slide condition shown in FIG. 8 a along withthe detector signals shown in 8 b.

The detection means is robust for the myriad of slide widths,thicknesses and surface finishes encountered in pathology labs.

The positions of slide present in slot are used for:

-   -   a. Determining which slides are present for scanning.    -   b. Determining the required x, y, z position to safely extract        the slide from the rack for scanning.

Thus, the invention provides a low-cost, high volume slide auto-loadersystem. In one embodiment, the autoloader according to the inventioncomprises:

-   -   a. A low cost, non-contact slide detector sensor system to        determine which slides are present.    -   b. Sensor system to accurately determine slide position for safe        slide extraction from rack to scanning system.    -   c. Set of standard slide racks for compatibility to other slide        processing equipment.    -   d. A means for detecting a misloaded slide which prevents errors        or damage to the slide from the slide grabber mechanism.

The slide auto-loader of the invention is advantageous in thatconventional scanner equipment either do not auto-detect slides orutilize expensive sensors systems to detect slides.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. A method for reconstituting an image comprising:dividing the image into a number of overlapping tiles captured in aseries of snapshots and reconstituting the image with a magnificationwithout substantial loss of accuracy; the method for reconstituting theimage further comprising: capturing at least two consecutive snapshotimages through a digital scanner; calculating stitch points between theat least two consecutive snapshots defining a first tile and a secondtile, respectively; wherein the first and second tile images have acommon overlap of at least N pixels between them; the method ofcalculating stitch points comprising: a. detecting and recording cornerpoints in a minimum overlap region and a maximum overlap region, wherebydetected corner points or a selected subset of corner points are sortedand maintained in a list; b. for each corner point selected in the firsttile, determining a set of possible stitch points in the second tile;wherein candidate points are selected using a goodness criterion; c.maintaining a pair of stitch points that have a matching score that isgreater than a defined or computed threshold; d. computing adisplacement between the tiles; e. calculating a confidence scoreassociated with the computed displacement; and f. determining astitching point for stitching the first and second tiles; and displayinga stitched image or storing a stitched image.
 2. The method of claim 1,wherein stitch points identify an area of overlap between the first tileand the second tile.
 3. The method of claim 2, further comprisingcropping the overlap area from one of the image tiles such that when thefirst and second tiles are put together an entire scene is renderedwithout significant loss of information.
 4. The method of claim 1,wherein the at least two consecutive snapshots are adjacent.
 5. Themethod of claim 1, wherein the at least two consecutive snapshotsoverlap by up to 300 pixels.
 6. A method for reconstituting an imagecomprising: dividing the image into a number of overlapping tilescaptured in a series of snapshots and reconstituting the image with amagnification without substantial loss of accuracy; the method forreconstituting the image further comprising: capturing at least twoconsecutive snapshot images through a digital scanner; calculatingstitch points between the at least two consecutive snapshots defining afirst tile and a second tile, respectively; wherein the first and secondtile images have a common overlap of at least N pixels between them; themethod of calculating stitch points comprising: a. detecting andrecording corner points in a minimum overlap region and a maximumoverlap region, whereby detected corner points or a selected subset ofcorner points are sorted and maintained in a list; b. for each cornerpoint selected in the first tile, determining a set of possible stitchpoints in the second tile; wherein candidate points are selected using agoodness criterion; c. maintaining a pair of stitch points that have amatching score that is greater than a defined or computed threshold; d.computing a displacement between the tiles; e. calculating a confidencescore associated with the computed displacement; and f. determining astitching point for stitching the first and second tiles; and displayinga stitched image or storing a stitched image, wherein stitch pointsidentify an area of overlap between the first tile and the second tile,and further comprising cropping the overlap area from one of the imagetiles such that when the first and second tiles are put together anentire scene is rendered without significant loss of information, andfurther comprising: a method for computing pixel correction factors,said method for computing pixel correction factors comprising: g.capturing a glass blank image or Air-blank image; h. smoothing the glassblank image or Air-blank image; i. determining a Max of R, G, B valuesover the entire glass blank image or Air-blank image; j. dividing atleast one of a R, G, and B value associated with each pixel in the glassblank image or Air-blank image by the corresponding Max of R, G and Bvalues, and generating a normalized image of the glass blank image orAir-blank image; and k. forming a reciprocal image by taking the inverseof the at least one pixel of the normalized image of the glass blankimage or Air-blank image, and generating pixel correction factors,wherein each of the at least one of a R, G, and B value associated witheach pixel in the reciprocal image corresponds to one of the pixelcorrection factors.
 7. A method for reconstituting an image comprising:dividing the image into a number of overlapping tiles captured in aseries of snapshots and reconstituting the image with a magnificationwithout substantial loss of accuracy; the method for reconstituting theimage further comprising: capturing at least two consecutive snapshotimages through a digital scanner; calculating stitch points between theat least two consecutive snapshots defining a first tile and a secondtile, respectively; wherein the first and second tile images have acommon overlap of at least N pixels between them; the method ofcalculating stitch points comprising: a. detecting and recording cornerpoints in a minimum overlap region and a maximum overlap region, wherebydetected corner points or a selected subset of corner points are sortedand maintained in a list; b. for each corner point selected in the firsttile, determining a set of possible stitch points in the second tile;wherein candidate points are selected using a goodness criterion; c.maintaining a pair of stitch points that have a matching score that isgreater than a defined or computed threshold; d. computing adisplacement between the tiles; e. calculating a confidence scoreassociated with the computed displacement; and f. determining astitching point for stitching the first and second tiles; and displayinga stitched image or storing a stitched image, wherein stitch pointsidentify an area of overlap between the first tile and the second tile,and further comprising cropping the overlap area from one of the imagetiles such that when the first and second tiles are put together anentire scene is rendered without significant loss of information, andfurther comprising: a method for computing pixel correction factors,said method for computing pixel correction factors comprising: g.capturing a glass blank image or Air-blank image; h. smoothing the glassblank image or Air-blank image; i. determining a Max of R, G, B valuesover the entire glass blank image or Air-blank image; j. dividing atleast one of a R, G, and B value associated with each pixel in the glassblank image or Air-blank image by the corresponding Max of R, G and Bvalues, and generating a normalized image of the glass blank image orAir-blank image; and k. forming a reciprocal image by taking the inverseof the at least one pixel of the normalized image of the glass blankimage or Air-blank image, and generating pixel correction factors,wherein each of the at least one of a R, G, and B value associated witheach pixel in the reciprocal image corresponds to one of the pixelcorrection factors and, further comprising: capturing an image of abiological specimen on a slide via a digital scanner; and multiplying atleast one of a R, G, and B value associated with each pixel of the imageof the biological specimen on the slide by the corresponding at leastone of R, G, and B pixel correction factors.
 8. A method forreconstituting an image comprising: dividing the image into a number ofoverlapping tiles captured in a series of snapshots and reconstitutingthe image with a magnification without substantial loss of accuracy; themethod for reconstituting the image further comprising: capturing atleast two consecutive snapshot images through a digital scanner;calculating stitch points between the at least two consecutive snapshotsdefining a first tile and a second tile, respectively; wherein the firstand second tile images have a common overlap of at least N pixelsbetween them; the method of calculating stitch points comprising: a.detecting and recording corner points in a minimum overlap region and amaximum overlap region, whereby detected corner points or a selectedsubset of corner points are sorted and maintained in a list; b. for eachcorner point selected in the first tile, determining a set of possiblestitch points in the second tile; wherein candidate points are selectedusing a goodness criterion; c. maintaining a pair of stitch points thathave a matching score that is greater than a defined or computedthreshold; d. computing a displacement between the tiles; e. calculatinga confidence score associated with the computed displacement; and f.determining a stitching point for stitching the first and second tiles;and displaying a stitched image or storing a stitched image, whereinstitch points identify an area of overlap between the first tile and thesecond tile, and further comprising cropping the overlap area from oneof the image tiles such that when the first and second tiles are puttogether an entire scene is rendered without significant loss ofinformation, and further comprising: a method for computing pixelcorrection factors, said method for computing pixel correction factorscomprising: g. capturing a glass blank image or Air-blank image; h.smoothing the glass blank image or Air-blank image; i. determining a Maxof R, G, B values over the entire glass blank image or Air-blank image;j. dividing at least one of a R, G, and B value associated with eachpixel in the glass blank image or Air-blank image by the correspondingMax of R, G and B values, and generating a normalized image of the glassblank image or Air-blank image; and k. forming a reciprocal image bytaking the inverse of the at least one pixel of the normalized image ofthe glass blank image or Air-blank image, and generating pixelcorrection factors, wherein each of the at least one of a R, G, and Bvalue associated with each pixel in the reciprocal image corresponds toone of the pixel correction factors, further comprising: capturing animage of a biological specimen on a slide via a digital scanner; andmultiplying at least one of a R, G, and B value associated with eachpixel of the image of the biological specimen on the slide by thecorresponding at least one of R, G, and B pixel correction factors,wherein the step of multiplying the at least one of a R, G, and B valueassociated with each pixel of the image of the biological specimen onthe slide by the at least one of R, G, and B pixel correction factorscorresponds to flat-field correction of the image of the biologicalspecimen on the slide.